Add Iceberg Quickstart page by ahmedabu98 · Pull Request #37739 · apache/beam

ahmedabu98 · 2026-03-02T20:53:21Z

http://apache-beam-website-pull-requests.storage.googleapis.com/37739/documentation/io/built-in/iceberg/index.html

Adding an Iceberg quickstart page to the Beam website

Includes examples for Hadoop Catalog (easy local development) and BigLake (cloud-scale development)

Includes the following SDKs:

SQL
Java
Python
YAML

gemini-code-assist · 2026-03-02T20:53:49Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new Apache Iceberg quickstart page to the Beam website, significantly enhancing the documentation for data lake integration. It provides practical, multi-language code examples for common Iceberg operations, from catalog setup to data manipulation, ensuring users can quickly get started with Iceberg I/O in Beam. Additionally, the changes include necessary updates to the website's interactive components to better display and organize these new code snippets and structured content.

Highlights

New Apache Iceberg Quickstart Page: A comprehensive quickstart guide for Apache Iceberg I/O has been added to the Beam website, providing detailed instructions and examples for integration.
Java and Python Iceberg Examples: New code examples in both Java and Python demonstrate how to configure Iceberg catalogs (Hadoop and BigLake), create tables, insert data, and query data using Beam's managed I/O.
Website UI Enhancements: The website's language switcher now supports SQL, and new generic tab and section shortcodes have been introduced to improve content organization and presentation for multi-language examples.
Java SDK Dependency Update: The sdks:java:managed dependency was added to the Java examples build configuration to support the new managed I/O operations for Iceberg.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

examples/java/build.gradle
- Added implementation project(":sdks:java:managed") dependency to support managed I/O.
examples/java/src/main/java/org/apache/beam/examples/snippets/transforms/io/iceberg/IcebergBeamSchemaAndRow.java
- New file: Defined Beam schemas and rows for various data types to demonstrate Iceberg compatibility.
examples/java/src/main/java/org/apache/beam/examples/snippets/transforms/io/iceberg/Quickstart.java
- New file: Provided Java quickstart examples for Iceberg, including Hadoop and BigLake catalog configurations, and managed write/read operations.
sdks/python/apache_beam/examples/snippets/snippets.py
- Added model_managed_iceberg function with Hadoop and BigLake catalog configurations, and managed write/read examples.
- Added model_managed_iceberg_data_types function demonstrating various data types for Iceberg.
website/www/site/assets/js/language-switch-v2.js
- Added 'SQL' as a supported language for the valueToTabTitle function.
- Initialized a new Switcher for generic tabs.
website/www/site/assets/scss/_syntax-highlighting.scss
- Added styling for .tab-switcher by extending .runner-switcher.
website/www/site/content/en/documentation/io/built-in/iceberg.md
- New file: The main Apache Iceberg I/O connector documentation page, including basics, quickstart guide (Hadoop and BigLake catalogs), and data type mappings across SQL, Java, Python, and YAML.
website/www/site/layouts/partials/section-menu/en/documentation.html
- Added "Apache Iceberg" to the I/O connector guides list.
- Simplified titles for other I/O connectors (e.g., "Apache Parquet I/O connector" to "Apache Parquet").
website/www/site/layouts/shortcodes/language-switcher.html
- Added sql as a language option for the language switcher.
website/www/site/layouts/shortcodes/section.html
- New file: Defined a Hugo shortcode for creating a section HTML element with a customizable class.
website/www/site/layouts/shortcodes/tab.html
- New file: Defined a Hugo shortcode for creating a div element styled as a tab, used for content organization.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

github-actions · 2026-03-02T22:44:20Z

Assigning reviewers:

R: @jrmccluskey for label python.
R: @Abacn for label java.
R: @shunping for label website.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

Abacn · 2026-03-09T20:55:40Z

Hi, is the PR ready for review (saw check failures, in case it is in draft)?

ahmedabu98 · 2026-03-10T15:33:34Z

@Abacn I'll take care of the failures, but yes it's ready for review!

github-actions · 2026-03-20T12:24:37Z

Reminder, please take a look at this pr: @jrmccluskey @Abacn @shunping

Abacn · 2026-03-20T17:09:05Z

examples/java/build.gradle

  implementation project(":sdks:java:io:google-cloud-platform")
  implementation project(":sdks:java:io:kafka")
  implementation project(":sdks:java:extensions:ml")
+  implementation project(":sdks:java:managed")


We should keep iceberg examples in examples/java/iceberg. There is an incoming risk Iceberg needs to be handled separately again as newer versions do not support Java11

good catch, just moved them to examples/java/iceberg

sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/IcebergScanConfig.java

Abacn · 2026-03-20T17:12:16Z

website/www/site/layouts/partials/section-menu/en/documentation.html

          <ul class="section-nav-list">
-            <li><a href="/documentation/io/built-in/parquet/">Apache Parquet I/O connector</a></li>
-            <li><a href="/documentation/io/built-in/hadoop/">Hadoop Input/Output Format IO</a></li>
+            <li><a href="/documentation/io/built-in/iceberg/">Apache Iceberg</a></li>


How are them ordered? Should we use alphabetical order or TOP IOs first + alphabetical order in remaining

Just wanted to put Iceberg up top for visibility. I left everything else in the same order

It largely seems to be in alphabetical order (with exception of CDAP being down there)

Abacn · 2026-03-20T17:13:44Z

website/www/site/content/en/documentation/io/built-in/iceberg.md

@@ -0,0 +1,433 @@
+---
+title: "Apache Iceberg"


Consider a different title? This affected Tab title on browser.

Changed to Apache Iceberg I/O connector

github-actions · 2026-03-25T12:29:45Z

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @tvalentyn for label python.
R: @chamikaramj for label java.
R: @kennknowles for label website.

Available commands:

stop reviewer notifications - opt out of the automated review tooling
remind me after tests pass - tag the comment author after tests pass
waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

Abacn

Thanks!

May need to rerun a bunch of tests before merge to get a clean signal

codecov · 2026-03-26T17:24:27Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.90%. Comparing base (26b18c6) to head (1236139).
⚠️ Report is 185 commits behind head on master.

Additional details and impacted files

@@              Coverage Diff              @@
##             master   #37739       +/-   ##
=============================================
+ Coverage     40.08%   56.90%   +16.81%     
  Complexity     3416     3416               
=============================================
  Files          1177     1178        +1     
  Lines        187315   187561      +246     
  Branches       3588     3588               
=============================================
+ Hits          75090   106725    +31635     
+ Misses       108833    77446    -31387     
+ Partials       3392     3390        -2

Flag	Coverage Δ
python	`80.09% <ø> (+40.35%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

add iceberg quickstart

9bb6a0f

github-actions bot added python java website examples labels Mar 2, 2026

ahmedabu98 added 3 commits March 2, 2026 15:57

add license

9de3ce4

spotless

942415d

adjustments

2509401

github-actions bot added the Next Action: Reviewers label Mar 2, 2026

ahmedabu98 added 4 commits March 3, 2026 11:13

lint

712d124

add missing io

d512ff1

add yaml hadoop config

d2e46c6

small fixes, and add support for python Timestamp

dbd2909

github-actions bot added io and removed io labels Mar 4, 2026

spotless

c3d802d

github-actions bot added io and removed io labels Mar 4, 2026

spotless

4a202d9

github-actions bot added io and removed io labels Mar 4, 2026

add public data quickstart

1017c25

github-actions bot added io and removed io labels Mar 8, 2026

yaml instruction

6401ba7

github-actions bot added the io label Mar 10, 2026

spotless

2973f62

github-actions bot added io and removed io labels Mar 10, 2026

ahmedabu98 added 3 commits March 10, 2026 11:50

spotless

611744c

fix spotbugs

1dca560

tweaks

27cb5b5

github-actions bot added the slow-review label Mar 20, 2026

Abacn reviewed Mar 20, 2026

View reviewed changes

github-actions bot added reassigned-reviewers and removed slow-review labels Mar 25, 2026

address comments

1438de1

ahmedabu98 requested a review from Abacn March 25, 2026 19:57

spotless

56f9438

Abacn approved these changes Mar 25, 2026

View reviewed changes

ahmedabu98 added 2 commits March 25, 2026 17:17

spotless

79dce55

remove unused dep

1236139

Conversation

ahmedabu98 commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist bot commented Mar 2, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

github-actions bot commented Mar 2, 2026

Uh oh!

Abacn commented Mar 9, 2026

Uh oh!

ahmedabu98 commented Mar 10, 2026

Uh oh!

github-actions bot commented Mar 20, 2026

Uh oh!

Abacn Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

ahmedabu98 Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Abacn Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

ahmedabu98 Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ahmedabu98 Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Abacn Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ahmedabu98 Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 25, 2026

Uh oh!

Abacn left a comment

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Mar 26, 2026

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ahmedabu98 commented Mar 2, 2026 •

edited

Loading

ahmedabu98 Mar 25, 2026 •

edited

Loading

Abacn Mar 20, 2026 •

edited

Loading